Fitting models to data: Accuracy, Speed, Robustness

In vision and machine learning, almost everything we do may be considered to be a form of model fitting. Whether estimating the parameters of a convolutional neural network, target tracking, computing low-dimensional representations of datasets, computing structure and motion from image collections, estimating parameters for an inference model such as Markov random fields, or pretty much anything else, it almost always boils down to minimizing an objective containing some parameters of interest as well as some latent or nuisance parameters. This tutorial will describe several tools and techniques for solving such optimization problems, covering a broad syllabus: vector and matrix calculus; nonlinear optimization algorithms such as gradient descent and its many variants, Quasi-Newton and Gauss-Newton derivates such as LBFGS and Levenberg Marquardt; modelling options for curves and surfaces; how to deal with missing data; how to deal with outliers, including how to optimize functions with robust kernels.